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Abstract 

In this paper, a class of optimization problems with nonlinear inequality con- 
straints is discussed. Based on the ideas of sequential quadratic programming 
algorithm and the method of strongly sub-feasible directions, a new super- 
linearly convergent algorithm is proposed. The initial iteration point can be 
chosen arbitrarily for the algorithm. At each iteration, the new algorithm 
solves one quadratic programming subproblem which is always feasible, and 
one or two systems of linear equations with a common coefficient matrix. 
Moreover, the coefficient matrix is uniformly nonsingular. After finite it- 
erations, the iteration points can always enter into the feasible set of the 
problem, and the search direction is obtained by solving one quadratic pro- 
gramming subproblem and only one system of linear equations. The new 
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algorithm possesses global and superlinear convergence under some suitable 
assumptions without the strict complementarity. Finally, some preliminary 
numerical experiments are reported to show that the algorithm is promising. 
Keywords: Nonlinear optimization, Sequential quadratic programming, 
Method of strongly sub-feasible directions, Global convergence, Superlinear 
convergence 
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1. Introduction 

In this paper, we consider the following nonlinear inequality constrained 
optimization problem 

min fn(x) 
(NCP) V ; 

s.t. fj(x)<0, j G / = {1,2, . . . , to}, 

where x G R n and the functions fj(x) : R n — > R [j G {0} U /) are all 
continuously differentiable. We denote the feasible set and gradients for 
problem (NCP) as follows 

X = {x G R n : fj{x) < 0, j G /}, 9j (x) = Vfjix), J G {0} U /. 

It is well-known that sequential quadratic programming (SQP) algorithms 
are acknowledged to one of successful algorithms available for solving problem 
(NCP) and have good superlinear convergence properties, they have been 
widely studied and investigated by many authors in l|, |2|, y|, |4|. 

The iterative process of the standard SQP algorithms is as follows. Let 
the current iteration point be x. Computing a search direction d by solving 
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the following quadratic programming (QP) subproblem 

min go(x) T d + \d T Bd 

s.t. h(x) + 9j{x) T d < 0, j El, 

where B G ^ nxn j s a symmetric matrix that approximates the Hessian of 
the Lagrangian function associated with problem (NCP) at (x, A), and A is 
a vector of nonnegative Lagrange multiplier estimates. Perform a search to 
determine a steplength t and let the next iteration point be x = x + td. 

However, in the classical SQP algorithms, there are two shortcomings: (i) 
(QP) subproblem may be inconsistent, i.e., the feasible set of (QP) subprob- 
lem may be empty; (ii) the Maratos effect [5| may occur, i.e., a full step of one 
can not be taken close to a solution of problem (NCP). In order to overcome 

nn 

disadvantage (i), various techniques have been proposed in [1|, |6jj. A popular 
way to overcome disadvantage (ii) is to use a higher-order direction, which is 



generated by solving a (QP) subproblem [l| or a system of linear equations 
(SLE) [7], or directly given by an explicit formula 8]. 

For SQP algorithms, feasible SQP (FSQP) algorithms are particularly 
useful for solving those problems arising from engineering design where the 
objective function f might be undefined outside the feasible set X. Another 
advantage of FSQP algorithms is that the objective function /o can be used 
as a merit function to avoid the use of a penalty function. In particular, 
Panier and Tits [I] present a FSQP algorithm in which the generated iter- 
ation points lie in the feasible set X. Two or three (QP) subproblems need 
to be solved at each iteration. In order to obtain the global convergence, 
they need to strengthen the requirement on the first-order feasible descent 
condition. The superlinear convergence rate is proved under the strict com- 



plementarity assumption. Zhu and Jian [9] further improve the algorithm 
in ll|. They introduce a new definition for the first-order feasible condi- 

n 

tion which is weaker than the first-order feasible descent condition in [1], 
and propose a new FSQP algorithm based on this new condition. The strict 
complementarity assumption is also necessary for obtain the superlinear con- 
vergence. 

One shortcoming of FSQP algorithms is that they are usually require a 



feasib 
work 



e starting point, while computing such a point is generally a nontrivial 



10] . In order to overcome this shortcoming, Polak et al. ll|] propose 



a combined phase I-phase II algorithm with arbitrary initial point for solv- 
ing problem (NCP). This algorithm becomes a method of feasible directions 
(MFD) Q when iteration point, ente r into the faaslb le set X. Jla n fnr- 



ther improve algorithm and propose a method of strongly sub-feasible 
direction in [131 ]. which not only unified automatically the operations of min- 
imization (Phase I), but also guaranteed that the number of the functions 
satisfying the inequality constraints is nondecreasing. Since their algorithms 
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only using the information of first-order derivatives, the algorithms in 
converge linearly at best. 

In this paper, motivated by the ideas in , we propose a new al- 

gorithm combining (QP) subproblem with method of strongly sub-feasible 
directions for solving problem (NCP). Unlike algorithm in 9], a descent di- 
rection d for /o at x is obtained by solving a (QP) subproblem which is 
always feasible. For obtaining the global convergence, a mere feasible direc- 
tion d is obtained by solving a SLE. Then, do is tilted by making a convex 
combination q — (1 — 0)do + (3d of do and d, where satisfies certain con- 
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dition. With the help of this convex combination, the global convergence 
is proved. In order to overcome the Maratos effect and obtain the super- 
linear convergence, a higher-order correction direction d\ is computed by a 
new SLE which has a common coefficient matrix with the previous SLE. 
Under the strong second-order sufficient conditions without the strict com- 
plementarity, the new algorithm is proved to be superlinearly convergent. 
Moreover, the initial iteration point is chosen arbitrarily, and after finite it- 
erations, the iteration points can always enter into the feasible set X. Finally, 
,some numerical results are reported to shown that the proposed algorithm 
is promising. 

At the end of this section, the main features of the proposed algorithm 
are summarized as follows: 

• the initial iteration point is arbitrary, and the number of satisfied con- 
straint functions is nondecreasing; 

• the objective function of problem (NCP) is used directly as the merit 
function, and the line search techniques are different from others; 

• at each iteration, the search direction is generated by solving one QP 
subproblem and one or two SLEs with the same coefficient matrix; 

• after finite iterations, the search direction is obtained by solving one 
QP subproblem and only one SLE, and the iteration points always enter into 
the feasible set X; 

• under some mild conditions without the strict complementarity, the 
proposed algorithm possesses global and superlinear convergence. 

This paper is organized as follows. In the next Section [21 we present the 
details of our algorithm and discuss its properties. In Sections [3] and HI the 
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algorithm is proved to possess global and superlinear convergence, respec- 
tively. In Section |5l some elementary numerical experiments are reported. 
Finally, some concluding remarks are given in Section [6j 

2. Description of algorithm 

To simplify the analysis, we use the following notations 



I~(x) = {] el: f 3 (x) < 0}, I + (x) = {3 el: f 3 \x) > 0}, (1) 
<p(x) = max{0, fj(x), 3 G 1} = max{0, fj(x), 3 G I + (x)}, (2) 



Assume that the following two basic assumptions for problem (NCP) are 
hold throughout this paper. 

Assumption 2.1. The functions fj (j G {0} U J) are all first- order contin- 
uously differentiable. 

Assumption 2.2. The gradient vectors {gj(x) : j G I(x)} are linearly inde- 
pendent for each x G R n . 

Remark 2.1. In Assumption \2.2\ the linearly independent gradients contains 
two parts: the one part is the gradients of the feasible constraint functions 
in active set, and the other part is the gradients of the maximal violated 
functions. This assumption is becoming the standard linearly independent 
constraint qualification (LICQ) only if the iteration point is feasible. More- 
over, this assumption plays a big role in the analysis of the following Lemmas 




[13 [U and Theorem\3J\ 
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For the current iteration point x k and an associated symmetric positive 
definite matrix B k , using the notations above, we introduce the following 



(QP) subproblem 14] 
(QPs) 



min g (x k ) T d + \d T B k d 



s.t. f 3 (x k )+g 3 (x k ) T d<0, Vjel, 
and we denote simply 

J fc - = r(x k ), 1+ = I + (x k ), I k = I(x k ), if k = <p(x k ). 

It is obviously that subproblem (QPs) has the following merits: 

(i) subproblem (QPs) is always feasible with feasible solution d = 0; 

(ii) subproblem (QPs) is a strictly convex program while B k is positive defi- 
nite, so it has an (unique) optimal solution; 

(hi) d is a solution of subproblem (QPs) if and only if it is a KKT point of 
subproblem (QPs). 

Let dp is an optimal solution of subproblem (QPs) at the k — th iteration, 
i.e., there exists a corresponding KKT multiplier vector \ k = (X k , j e 7) 
such that 

g Q {x k ) + B k d k + Y.^9 3 ix k ) = ^ 

f 3 {x k ) + g 3 {x k ) T d k < 0, X k > 0, Af(£(z*) + g 3 {x k ) T d k ) = 0, Vj e I. 

(4) 

From PJ) and the KKT condition for problem (NCP), the following lemma 
holds immediately. 

Lemma 2.1. x k is a KKT point for problem (NCP) if and only if (d k , if k ) — 
(0, 0). 



On one hand, in view of d = is a feasible solution and g?q is an optimal 
solution for subproblem (QPs), respectively, it follows that 

go(x k fd k + ^d k ) T B k d k <0, (5) 

and if c?q 7^ 0, together with the positive defmiteness of B k , (jHJ) imply that 

g (x k ) T d k <-^(d k ) T B k d k <0, 

i.e., dp is a descent direction for problem (NCP) at x k . 

On the other hand, d^ may be not a feasible direction for problem (NCP) 
at the feasible iteration point x k . Even when d^ is a feasible direction, a line 
search may not allow a full step of one to be taken in a neighborhood of an 
optimal solution and thus superlinear convergence may never take place. In 
order to get an improving direction and taking into account that x k may be 
infeasible, we first propose a new SLE 

B k N k \ I ,/ \ I 

(6) 





Nl ~D k \h \ -{W\\ + tf)rn 



to generate an updated direction d k , where G R n , w = (1, 1, . . . , 1) T G R 
a G (0, 1) and 



in 



N k = (g,(x k ), j G I), D k = di ag (D k , j g /), 
D k = \fj(x k )\(\fAx k ) +9j(x k ) T d k \ + H\\), j G /. 



(7) 



From (jBJ) and (JTj), it follows that 

g j (a*) T d k = -\\4\\-<p%, VjG4, 

i.e., d k is a mere feasible direction. 

The following lemma describes the solvability of SLE (jHJ), its proof is 



1.2 in Q 



similar to Lemma 2.2 in 1151 and is omitted here. 
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Lemma 2.2. Suppose that Assumptions \2. 1\ and \2.2\ hold and B k is positive 
definite. Then, the coefficient matrix V& defined in is nonsingular and 
has a unique solution. 

Then, in order to yield an improving search direction at iteration point 
x k (feasible or infeasible), we consider a convex combination of d$ and d k as 
follows 

q k = (1 - (3 k )d k + (3 k d k , (8) 

where 0k is the maximum value of G [0, 1] that satisfies the following 
relationship 

g (x k ) T q k = (1 - k )g Q {x k ) T d k + k g o {x k ) T d k < 9g Q {x k ) T a% + rf, (9) 

where the positive parameter 9 < o. Taking into account that g^{x ks ) T d\ < 0, 
it follows that (5 k can be yielded by the following explicit formula 

1, if g (x k ) T d k < g (x k ) T d k . 

Remark 2.2. From it follows that the rate of increase of the objec- 
tive function fo at point x k along direction q k is bounded from upper by 
9g Q (x k ) T d k l + ipl, and it is just so ensuring the direction q k possesses ex- 
cellent convergence. In addition, the parameter 9 e (0, o~) plays important 
roles in avoiding Maratos effect for analyzing the request fTfy in the algo- 
rithm as well as forcing the iteration points always get into the feasible region 
after a finite number of iterations. These can be seen in the latter analysis, 
e.g., Lemma \3. 1\ and Theorems 4-S\ \4-3\ as well as 4^4 



From the relationships of (jHJ), ([9]) and ([6]), the following lemma can be 
proved easily. 
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Lemma 2.3. Suppose that assumptions in Lemma \2.S\ hold. Then 
(i)9o(x k ) T q k < -le(d k ) T B k d k 0+( p e h ; (ii) 9j (x k ) T q k < -(3 k (\ \d k \ \ + ^), Vj G 

From Lemma 12. 3^ it holds that q k is an improving direction either for 
problem (NCP) or for the maximal violated constrained function <p(x). In 
order to overcome the possibility of Maratos effect, a suitable higher-order 
correction direction must be introduced by an appropriate approach. Addi- 
tionally, taking into account avoiding the strict complementarity condition, 
we introduce another SLE 




(11) 





-(\\d k \\T + <p%)w-F(x k + d k ) 
to yield a higher-order correction direction d k , where r G (2, 3) and 

F(x k + d k ) = (fj(x k + d k ) - f,(x k ) - 9j (x k ) T d k , j G /). (12) 

Remark 2.3. The right- hand- side of 07]) (in particular, the introducing 
of F(x k + <ig) ) as well as d k play an important role in the discussion of 
superlinear convergence in Section^ without the strict complementarity, these 
can be found in the proofs of Lemma [XT] and Theorem \A.2\ In traditional 

nnh f- 1 

analysis Jj, \M, lid /. d\ must satisfy \\d k \\ = 0(||<ig|| 2 ) that is called second- 
order correction. In this paper, the term is added, so the relation between 
d\ and d^ will be different from the traditional forms, see Lemma \i.l\ i). 



Now, based on the analysis above, we can present our algorithm as follows. 
Algorithm 2.1. 
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Parameters: 7, T], £ G (0, 1), < < er < 1, < g < cr, £, ( > 0, a G 
(0,0.5), p > 1, 5 > 2, r G (2,3). 

Data: x° G i? n , a symmetric positive definite matrix Bq G R nxn . Set 
fc := 0. 

Step 1. Solve subproblem (QPs) to get a (unique) solution d§ and the 
corresponding KKT multiplier vector X k = (X k , j G J). If (g?q, <^fc) = 0, then 
x k is a KKT point for problem (NCP) and stop. 

Step 2. Compute the correction direction d k by solving SLE (II ip with a 
solution (d k , h k ), and let d k = d^ + d\. If 

^o(x fc ) T 4<Cmin{-||rf fc || 5 , + (13) 

then go to Step 3; otherwise, go to Step 4. 
Step 3. Let t = 1, 

(a) if 

f (x k + td k ) < f (x k ) + atg (x k ) T d k + p{\ - a)t<pl 
< f j (x k +td k )< ( p k -at(\\d^\\' r + ( p%), JG/+, (14) 
^ /,(a; fc + td fc )<0, jG4~, 

is satisfied for the current value t, then let % = t, go to Step 6; otherwise, go 
to (b) below. 

(b) Reset t := \t. If t < e, then go to Step 4; otherwise, repeat part (a). 
Step 4. Solve SLE ([7]) to get (d k , h k ), and compute 0k according to (fTOj) . 

then obtain the direction q h by (jSJ). 

Step 5. Compute the steplength tk which is the first number t in the 
sequence {l,r],r] 2 , . . .} that satisfies the following inequalities 

h{x h + tq k ) < f (x k ) + 1 tg {x k ) T q k + p(l - 7 )t^, (15) 
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fA^ + t<i k )<Vk-Ttp k (\\d*\\ + ipl), jel+, (16) 
/•,•(** + t g *)<o, jg/;, (17) 

then let d k = q k . 

Step 6. Compute a new symmetric positive definite matrix -B^+i by some 
suitable techniques, set x k+l = x k + £/s<i fc , := k + 1, and go back to Step 1. 

Remark 2.4. From the mechanism of Algorithm \2.1\ we know that when 
ifk 7^ 0, the value of the maximal violated constrained functions (namely 
ifk) is strictly decreasing, moreover, the amounts of decreasing have a close 
relation with \ \d k \ \ + ipk- Thus, the iteration points can approach the feasible 
region as soon as possible. In particular, they can always getting into the 
feasible set after a finite number of iterations. 



Remark 2.5. It is known that the role of the request ( T7jj) is to restrict the 
increasing rate of the objective function /o at point x k along direction d*. 
But, theoretically speaking, under the uniformly positive definite assumption 
(see Assumption \3.1\ below) on the sequence {-Bfe} of matrices, the request 
(Q2|) does not influence any theory analusisof Algorithm \2. 1\ this can be seen 
in the latter analysis. However, the request lp~3\) still has some influence on 
the numerical effect of Algorithm \2.1\ From the process of numerical experi- 
ments, it seems to be that, for small-scale problems, the numerical results of 
Algorithm \2. 1\ with the request ( fJ5j) are better than the case of ignoring this 
request, and the case is inverse for middle-large-scale problems. In addition, 
the role of the exponents 5 > 2 and g < a is to ensure, under suitable as- 
sumptions, that the request lp~3\) is always satisfied when the iteration point 
x k is close sufficiently to a KKT point, see Lemma \J^.l\ iii). 
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Remark 2.6. There are two cycles between Step 1 and Step 6, i.e., cycle 
I: Steps 1-2-3-6, and cycle II: Steps 1-2-4-5-6. Obviously, if the algorithm 
successfully perform cycle I, the cost of computation is relatively small. For- 
tunately, under suitable assumptions, we can prove that Algorithm \2. 1\ always 
performing cycle I after a finite number of iterations. 

The lemma given below indicates that the line search in Step 5 of Algo- 
rithm 12.11 is well defined. 

Lemma 2.4. Suppose that Assumptions \2.1\ and \2.2\ hold. Then, if Algorithm 
\2.1\ does not stop at Step 1, i.e., (d$, <fk) (0, 0), the line search < f73]) -(fT7[) 
can be terminated after a finite number of computations. 

Proof. Suppose that (d k , <fk) 7^ (0, 0) at the k-th iteration. From (fTDl) . it 
follows that /3k > 0. 

(1) Analyze the inequality (j!5p . From Taylor expansion and Lemma 
I2.3( i). it follows that 

a h (t) = fo(x k + tq k ) - f (x k ) - 1 tg G {x k ) T q k - p(l - 7 )t<p 9 k 

= (1 - 1 )tg {x k ) T q k - p(l - i)t<& + o(t\\q k \\) (18) 
< -19(1 - !)t(d k ) T B k d k - (p - 1)(1 - 7 ) tV i + o(t\\q k \\). 

This together with 0,7 e (0, 1), p > 1 and (a%) T B k a% + (p e k > shows that 
flfc(t) < holds for t > sufficiently small. 

(2) Analyze the inequalities ffTB]) . 

(i) For j 6 Ij + H Ik, it follows that fj(x k ) = (fk, expanding fj(x h + tq k ) 
around x k , and combining Lemma l2~3T ii) . for 7 G (0, 1) and t > sufficiently 
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small, we obtain 

f,(x k + tq k ) -<Pk + -ytl3 k (\\d*\\+<p%) 

= tg 3 (x k ) T q k + ytfiu (| | d k \ \ + tf) + o(t\ \ q k \ \ ) 
<-(l-7)*A(l|dg||+^) + o(t||g*||) 
< 0. 

(ii) For j G l£\h, we have fj(x k ) — <fk < 0, and furthermore, 

\im (f,(x k + tq k ) - <p h + 7fft(||dSll + <pl)) = fjtf) ~ <Pk < 0, 
t— >o+ 

which implies the inequalities (fT6j) hold for t > sufficiently small. 
(3) Analyze the inequalities (ITT)) . 

(i) For j G JjT fl expanding + at x fc , and taking into account 
Lemma l2.3( ii). for t > sufficiently small, it follows that 

fjtf+tf) = t gj (x k ) T q k + o(t\\q k \\) < - t (3 k (\\d k \\ + V l) + o(t\\q k \\) < 0. 

(ii) For j G I k \Ik, we have fj(x k ) < 0. So, from Assumption 12. 1^ it 
follows that fj(x + tg fc ) < 0, for t > sufficiently small. 

Summarizing the analysis above, we conclude that there exits a > 
such that the line search (fT5|) - (fT7|) satisfies for all t G (0, and the given 
conclusion holds. □ 

At the end of this section, based on the line search conditions (I14j) . (Tl~6l) 
and (ITT)) , we can easily get the following lemma. 



Lemma 2.5. Suppose that Assumptions \2. 1 \ and \2.2\ hold. Then, 

(i) for each k, I k C holds, so, if there exists an iteration index fc suc/i 

£/icrf G A ; ie. ; <pk = 0, then x h G X for all k > k , and {fo{x k )}k>k ^ s 
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decreasing; 

(ii) ifx k £ X andx k+1 £ X, thenip k+1 < ip k -t k max{a(| | T +y^), T/Sfc(|Moll+ 
< W 

(Hi) for k large enough, the subsets 1^ and l£ can be fixed, i.e., 1^ = I 
and J+ = /+. 

Remark 2.7. From Lemma WTbY i) and (ii), it follows that exactly one of the 
following two cases takes place: 

Case A: There exists an iteration index ko such that (p ko = 0, then Lp k = 
for all k > k ; 

Case B: For any k = 0, 1, 2, . . ., ip k > and ip k+ i < <p k . 

3. Global convergence 

In this section, we will establish the global convergence of Algorithm 12.11 
When Algorithm 12.11 stops at x h , it follows that the iteration point x k is a 
KKT point for problem (NCP) from Lemma [27TI and Step 1 of Algorithm [27TJ 
Now, suppose that an infinite sequence {x h } of iteration points is generated 
by Algorithm 12. 1} and we will show that every accumulation point x* of 
{x k } is the KKT point for problem (NCP). For this purpose, the following 
assumption is necessary. 

Assumption 3.1. The sequence {B k } of matrices is uniformly positive def- 
inite, i.e., there exist two positive constants a and b such that 



a\\d\\ 2 < d T B k d < b\\d\\\ Wd e R n , Vfc. 



(19) 



Denote the active set for subproblem (QPs) by 



J k = {jEl: f,(x k )+ gj (x k ) T d k =0}. 



(20) 
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Suppose that x* is a given accumulation point of {x k }. In view of 
l£, Ik i Jk all being subsets of the finite set I and Lemma l2.5( iii). we can 
assume without loss of generality that there exists an infinite index set K 
such that 

x k ->• x*, I- = /-, 1+ = /+, J k = J, <p k ->• ^ = ^(z*), VA; G K (21) 
Based on the above conditions, we have the following lemmas. 

Lemma 3.1. Suppose that Assumptions \2. 1\ \2.2\ and \3.1\ hold. Then 

(i) sequence {dfyx — {d^ : k G K} is bounded; 

(ii) there exists a constant c > such that IIV^ 1 )! < c for all k G K, where 
Vk is the coefficient matrix defined by (EJ); 

(Hi) sequences {d k }x, {d k }x, {q k }K and {h k }K are all bounded. 

Proof, (i) First, from lim x k = x* and the continuity of go(x), there exists a 

k£K 

constant c > such that ||go(^ fc )|| < c holds for all k G K. Then, from (jHj) 
and Assumption 13 .lj it follows that — c [ | cZ§ 1 1 + ^ <x 1 1 1 1 2 < 0, VA; G which 
implies the boundedness of {dfyx- 



(ii) The proof of conclusion (ii) is similar to Lemma 3.1 in 15|, thus it is 
omitted here. 

(iii) According to (jSJ), (jTTj) and parts (i) and (ii), it follows that {d k }x, 
{d k }x and {h k } are all bounded. Furthermore, from (JEJ) and G [0, 1], we 
obtain the boundedness of {q k }x- □ 



Lemma 3.2. Suppose that assumptions stated in Lemma \3.1\ hold. Then 
lim(4,^) = (0,0) 
K is defined by Wf\, 



lim(c?n, Vk) = (0,0) and lim d k = lim q k = lim = 0, where the index set 
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Proof. We first show that lim(<iQ, ipk) = (0,0). Suppose by contradiction 
that there exists an infinite index set K C K and a constant e > such 
that 11(^0,^)11 > e, £ -K" • So, there exists an infinite set if C X such 
that o% -> do, <Pfc ->■ V 5 *) -Sfc -> -B*, fee if". Thus, in view of (|5]), (JH]) and 
G (0, 1), we have 

(6 - l)9o(x k ) T d k + <p° > 1(1 - 6)a\\a%\\ 2 + V { k ^ 1(1 - 0)a| |cQ| | 2 + ^ > 0, 
which further shows that 

li^ii 2 + ^ feJ ^'lMSii 2 + ^>o, 11411 + ^^' IKH + ^>o. 

This together with (g?q, ifk) 7^ and ffTUl) implies that there exists a constant 
e > such that 

/3 fc >e, |MS|| a + rf>e, Poll + ^ > VTc G K" . (22) 

The rest proof is divided into two steps as follows. 

Step A. Show that there exists a constant t > such that the steplength 
t k >i for all k G K" . 

From the mechanism of Algorithm 12.11 we know that it is sufficient to 
prove that the inequalities (jT5]) - (fTT|) are satisfied for all k G K" and t > 
small enough. 

(1) Analysis the inequality ffT5l) . From the boundedness of {o%]k and 
(HHD, (USD as well as (|22L we have 

a*(f) < -(1 - 7 Mf «) T ^ fe + (p - 1M) + o(t) 

< _(i _ 7 )t(ia#||4|| 2 + (p - %*) + o(t) 

< -(1 - 7)tmin{ia^, p - 1}(|KI| 2 + + o{t) 

< -(1 - 7 )te min{§a#, p - 1} + o(t) < 
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holds for t > small enough and all k £ K . 

(2) Analysis the inequalities f|T6|) . 

(2-i) For j £ I + and ^(x*) = <p m , from © and it follows that 
D k = -<p k \{\fi{x k ) + gj (x k ) T d k -p k \ + \\d k \\) ^ 0, A; £ K". 

Therefore, from Taylor expansion, (jSJ), (EI) and the constraints of subproblem 
(QPs), for t > sufficiently small and all k £ X , we obtain 

f j (x k + tq k )-i Pk + 1 k (\\d k \\ + ^ k ) 

= f,(x k ) + t 9j (x k ) T q k -<p k + 7*^(||dg|| + ^) + o(t) 

= - <p k + t(l - k ) gj (x k ) T d k + tf3 k9] (x k ) T d k + jt/3 k (\\d k \\ + ifl) + o(t) 

< (1 - f(l - &))(/;(**) - - t/3 fc (|Kll + ^) + *7&(l|dg|| + f%) + o(t) 
<-(l- 7 )tft(||d*||+^) + o(f) 

< -e 2 (l - 7 )t + o(t) < 0. 

(2-ii) For j £ J + and fj(x*) < y?*, we have 

/.V + - ^ + 7^(11411 + = - + 0(t) 

<Ufi(x*)-<P*) + 0(t) 
< 

holds for all k £ fT" and £ > sufficiently small. 

(3) Analysis the inequalities (fT7|) . Similar to the analysis for the inequal- 
ities ( Fl6|) . one can show that ( ITT)) hold for all k £ X" and t > small 
enough. 

Summarizing the analysis above, we conclude that there exists a t > 
such that t k >t for all /c £ K". 

Step B. Use > t > (A; £ if ") to bring a contradiction, and the 
discussion is divided into two cases. 
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Case I. Suppose that there exists an iteration index fc such that p k o = 0. 

Then {fo(x h )}k>k is decreasing. Combining lim fo(x ) = fo(x*), it follows 

keK" 

that lim fo(x k ) = fo(x*). On the other hand, taking into account the first 

fc— >+oo 

inequality of (03), flU, Lemma Qi), (JT9]) and ([22]) as well as p fe = (Vfc > 
fco), it follows that 

/ (x fc+1 ) < /o(z fc ) + tk max{ag (x k ) T d k , ^g {x k ) T q k } 

< f (x k ) + t k max{- |aa||^|| 2 , -\9^a\ \d%\ | 2 } 

< fo(x k ) — temin{|aa, |#7a}, Vfc > fco, G if . 

Thus, passing to the limit fc G if and fc — > +oo in the inequality above, we 
can bring a contradiction. 

Case II. Suppose that p k > for each fc. Then {y?fc}fc>o is decreasing. 
Combining lim ip k = ip*, one knows that lim ip k = tp*. On the other hand, 

keK" ' fc-s-+oo 

taking into account the second inequality of (JHJ), ([IS]) . Lemma [2T3lf ii) . (]T9]) 
and ([22]) as well as > (Vfc G if ), it follows that 

Pk+i <Pk-t k m&x{a(\\d k \\ T + p%), 7&(ll4l I + 

< tpk — temaxja, 7e}, Vfc G if" , fc large enough. 

Then, passing to the limit fc G if and fc — > +oo in the above inequality, we 
can also bring a contradiction. 

Up to now, we have finished the proof of lim^, pk) = (0, 0). 

Finally, by Lemma EITJii), ®, ® and lim (4, p k ) = (0, 0), it follows 

k£K 

that lim c? fc = lim q k = lim d k = 0. □ 

fce-ft: fcex feeds' 

Now, we present the main result of this section. 

Theorem 3.1. Suppose that Assumptions \2. 1\ \2.2\ and \3.1\ hold. Then Al- 
gorithm \2.1\ either stops at a KKT point x k for problem (NCP) after a finite 
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number of iterations or generates an infinite sequence {x k } of points such that 
each accumulation point x* (if it exists) of {x k } is a KKT point for prob- 
lem (NCP), furthermore, there exists an index set K such that {(x k ,X k )}K 
converges to the KKT pair (x*, A*) for problem (NCP). 

Proof. Choose an infinite index set K such that (|2ip holds, and let matrix 
Rk — {9j{ xk )i j £ J)- I n v i ew of (x k ,dQ,(p k ) — > (a;*, 0,0), k G K, we can 
conclude that J C I(x*) = {j G / : fj(x*) = 0}, and this together with 
Assumption 12.21 shows that R^Rk is nonsingular for k G K large enough, 
since Rk — > R* — (gj(x*), j G J). Again, from the KKT condition (j4]), we 
have 

g (x k ) + B k d k + RkX k j = 0. 
Thus, for k G K sufficiently large, we have 

X k = -(RlR h )- 1 Rl(g (x k ) + B k d k ) -« R*)- 1 Rj g (x*) = f A}. 
If we denote the multiplier vector A* = (A}, On/), then lim A fc = A*. 

k£K 

Furthermore, passing to the limit k G K and k — > oo in (jl}, it follows that 

g (x*) + N*\* = 0, fjix*) < 0, A* > 0, /,(x*)A* = 0, j G J, 

which shows that (x*, A*) is a KKT pair for the problem (NCP). □ 

Remark 3.1. The global convergence Theorem \3. II shows that if the sequence 
x k generated by the proposed Algorithm \2. II possesses a limit point x* , then 
x* is a KKT point for problem (NCP). 

4. Superlinear convergence 

In this section, under some suitable assumptions, we first prove that Algo- 
rithm possesses strong convergence, and the iteration points always enter 
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into the feasible set X after a finite number of iterations. Subsequently, the 
superlinear rate of convergence is established without the strict complemen- 
tarity. For these purposes, we further make the following assumption. 

Assumption 4.1. (i) The functions fj(x) (J E {0}U/) are all second-order 
continuously differentiable. 

(ii) The sequence {x k } generated by Algorithm \2.1\ is bounded, and possesses 
an accumulation point x* , such that the KKT pair (x*, A*) satisfies the strong 
second-order sufficient conditions (SSOSC for short), i.e., 

d T V 2 xx L(x*, \*)d > 0, Vd E FT, d^O, gj{x*fd = 0, j E I+, 

where L(x, A) = f (x) + £ \fj{?), 1+ = {j E I : A* > 0}. 

jer 

Remark 4.1. On the one hand, in some previously proposed SQP-type al- 
gorithms \\, Q/, to get the superlinear convergence of the proposed algorithm, 
one has to ensure relation = I(x*) holds for k large enough. And the strict 
complementarity (i.e., = I{x*)) is an very important condition for ensur- 
ing Jk = I(x*) holds. On the other hand, it is well-known that the SSOSC 
is equivalent to the second-order sufficient conditions (SOSC for short) un- 
der the condition of the strict complementarity, however, this condition is 
hard to verify in practise, and the positive space (i.e., the critical directions 
set) of the SOSC is smaller than the SSOSC 's. In this paper, in order to 
obtain the strong and superlinear convergence of Algorithm \2.1\ combining a 
slight strong assumption SSOSC, we can still avoid the Maratos effect under 
It Q Jk Q I(x*) (see Lemma J^.l, Theorem 4-2fy - 



First, under the stated assumptions, we have the following theorem. 
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Theorem 4.1. Suppose that Assumptions \2. 2\ \3.1\ and\4-l\ hold. Then 



(i) lim g?q = lim (ij = lim <i fe = lim g fc = lim d k = and lim ||s fc+1 - 
= 0; 

(ii) lim x fc = x*, i.e., Algorithm \2.1\ is said to be strongly convergent in this 

k— >+oo 

sense; 

(Hi) lim Lp k = <f(x*) = 0, lim A fc = A*. 

k— >+oo k— ¥+00 

Proof, (i) Since {x k } is bounded, from Lemma [3T2l it follows that any subse- 
quence of {(<io, d k , d k , q k )} must possesses an accumulation point (0, 0, 0, 0) e 
R An , which implies lim (dQ,d k ,d k ,q k ) = (0,0,0,0). Furthermore, from the 
mechanism of Algorithm 12.11 we obtain 

lim d k = 0, lim ||x fc+1 -x fc ||= lim \\t k d k \\ = 0. 

k—t+oc fe— >+oo k— >+oo 

(ii) First, under Assumptions 12.2 1 and 14.14 by Proposition 4.1 in jl|, it is 
known that the given limit point x* is an isolated KKT point of problem 
(NCP). Therefore, we know from Theorem 13.11 that x* is an isolated limit 
point of {x h }, this combining ||a; fe+1 — x k \ \ — > 0, it follows that x k — > x* (The 
details can be found in [14j|). 

(iii) In view of Theorem 13.11 we know that the given accumulation point 
x* is a KKT point for problem (NCP), thus it follows tp(x*) = 0. Further- 
more, combining the monotone property and boundedness of {(fk}, we have 

lim ifk = (p(x*) = 0. Moreover, from the proof of Theorem 13.11 and parts 

k— >+oo 

(i) and (ii), one can conclude that each accumulation point of sequence {A^} 
is a KKT multiplier associated with x*, this together with the uniqueness of 
the KKT multiplier implies that lim A fc = A*. □ 

fe— >+oo 

The relations established in the following lemma are very important in 
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the subsequent discussion. 



Lemma 4.1. Suppose that Assumptions \2. 4 IJ.il and \4-l\ all hold. Then 



(%) \\d\\\ = o(\\d*\\*)+o(rf), ll^ll 2 = o(||4|| 4 )+o«) ; mw = o(||4ll 2 )+ 

OH); 

(ii) J+ C J k C /(a;*) /or /arge enough; 

(Hi) the relationship [T3) is satisfied for k large enough. 

Proof, (i) In view of F(x fc + d*) = C(||c?q|| 2 ), the proof is elementary from 
([TT]) and Lemma I3.1( ii) as well as Theorem 14.11 

(ii) For j I(x*), i.e., fj(x*) < 0. Since lim (x fc ,<io) = (x*,0), there 

fe— >+oo 

exists a constant £ > such that fj(x k ) = fj(x k ) < — £ < holds for k 
large enough. It then follows that fj(x k ) + gj(x k ) T dQ < — 1£ < for large 
enough, which shows that j Jk, thus Jk C J(x*). Furthermore, it follows 
from Theorem HJ] that lim A, + = A*+ > 0, so, A ^+ > and 1+ C J fc hold 
for A; sufficiently large. 

(iii) From fl5]) and Assumption 13. 1[ we only need to show that 

-^||4|| 2 <Cmin{-||4H 5 , -\\d k \\ S } + ^l 

holds for k large enough. 

First, in view of S > 2 and Lemma I3T21 it follows that 



-^ll4ll 2 <-Cll4ll 5 <-CIMo fc H 5 + ^, (24) 

for k sufficiently large and (, a > 0. 

Second, in view of \\d k \ \ = + ^i|| < I Moll + I Mi II) <5 > 2 , part (i) and 



Lemma [3.21 we obtain 
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for k large enough. This together with 5 > 2 and g < o implies that 

-CIMT + ^ + ilKII 2 >-CII4ll 5 + fH4ll 2 + o(lKII 2 ) + ^ + oM 



2 

> 



a IKII 2 + °(IKII 2 ) + ^ + %^ 



holds for k large enough, i.e., 

- ^IMoll 2 < -C\\d k \\ S + £<4, for A; large enough. (25) 

Finally, combining (124]) . f l25|) and fl5]), the conclusion (iii) holds for k large 
enough. □ 

To ensure that the steplength t k = 1 for k large enough without the strict 
complementary assumption, an additional assumption as follows is necessary. 

Assumption 4.2. Suppose that the KKT pair (x*, A*) and the matrix B k 
satisfy 

miM^,y)-B k )d k \\ = o(\\d k \\). 



Remark 4.2. According to Theorem 4-4 > H holds that 



Assumption 5 \\{V 2 xx L{x k ,\ k ) - B h )d k \\ = o(\\d k \\). 

Theorem 4.2. Suppose that Assumptions Iff. 4 \3.1l \4-l\ and \4-2\ are all sat- 
isfied. Then, the inequality [Lfy in Step 3 is always satisfied for t = 1 and 
sufficiently large k. Therefore, Step 4 and Step 5 are no longer performed in 
Algorithm \2.1\ and Algorithm \2.1\ always performs cycle I. 

Proof. First of all, in view of Lemma T4.1 ( iii). it is sufficient to prove that the 
inequality f[T4"l) holds for t — 1 and sufficiently large k. 
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Discuss the second group of inequalities and the last group of inequalities 
of dHJ). 

For j £ I{x*), i.e., fj(x*) < 0, in view of (x k , d k , d k , (p k ) -> (x*, 0,0,0) (k ->■ 
+00), we obtain d k = <ig + d k — > (k — > +00), thus, we conclude that the 
second group of inequalities and the last group of inequalities of flHJ) are 
both satisfied for t = 1 and k large enough. 

For j G I(x*), since lim fj(x k ) = fj(x*) = and lim ifk = as well 
as ([2D and ([7|), it follows that 

/•(x fc )^0, £>J->0, D k = o{\j 3 {x k )+ g] {x k ) T d k \) + o{\\d k \\). 

On the other hand, from (11 II) and Lemma l4.1( i). we have 

9j (x k ) T d k = -\\d k \Y -vl- f,(x k + d k ) + f 3 \x k ) + g 3 (x k ) T d k + D k h k j 
= -Kir - ^ - + dg) + fj(x k ) + <fc(x fc ) T d* 
+o(|/ i (x fe ) + gj (x k ) T d k \) +0(||4H 3 ) +o(tf). 

(26) 

Then, from Taylor expansion, Lemma l4.1( i). ([26]) and r G (2,3), we obtain 

f^ x k + d k) = + d g) + + d kf d k + 0(| |4| | 2 ) 

= /,(x fc + dg) + fc(s*) T dJ + 0(1 141 1 3 ) + °M 

= -H4i| T - ^ + /i(^) + <&(**) T d8 + o(|/,(x fc ) + 9j (x k ) T d k \) 

+o(||4|| 3 ) + (^), 

(27) 
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fj(x k +d k " 



which further implies that 

' - - \fj(x k ) + g i (? k ) T d*\ +0(15(1*) + gj(x k ) T 41) 

+O(\\d*\\ 3 )+o(<p%), Xjel(x*)nl-, 
-\\d k \r -tf + Vk- \fj(x k ) + 9j(x k ) T d k Q \ + o(\f 3 (x k ) + g 3 {x k ) T d k \) 
+0(\\d k \\ 3 ) + o(<pi), if jel(x*)nl+. 

(28) 

This shows that fj(x k + d k ) < 0, for j 6 /(s*) fl J - and large enough. 

For j e I«)nl+ from ([2Sj) and r G (2, 3) as well as a e (0, f ), it follows 
that 

= -(i - a)(iidgir + ^) - i/i(ir*) + ^(xT4i (29) 

+o(|/ i (x fc ) +^(x*) T d*|) +o(^) +0(|K|| 3 ) 

< o. 

Thus, summarizing the analysis above, we have proved that the inequality of 
(JUJ) except the first one is satisfied for t — 1 and k large enough. 

Finally, we will show that the first inequality of (j!4p holds for t — 1 and 
large enough. From Taylor expansion and Lemma l4.1( i). we have 



A fc 4 f (x k + d k ) - f (x k ) - ag (x k ) T d k - p{\ - a) V { 

= g (x k ) T d k + \{d k ) T VlJ {x k )d k - ag (x k ) T d k - p(l - a)^ k + o 
= 9o(x k ) T (d k + d k ) + Ud k ) T VlJ (x k )d k - ag (x k ) T d k - p(l - a)<p{ 



+o(H\\ 2 )+o(<pl). 



Then, from the KKT conditions (jlj) and Lemma l4.1( i). we have 



(30) 



g (x k ) T d k = -(d k ) T B k d k - \Wf< 



(31) 
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g (x k ) T (d k + d k ) = -(d k ) T B k d k - £ A^(*TK + d\) + o(\\d k \\*) + o(<p%). 

(32) 

For j G J k C J(x*), it follows that / i (x fe ) + Vfj(x k ) T d k = 0, which 
together with ( 1281) and </>fc = o(^) implies 

+ d *) = _||d*||r _ ^ + o(||dS|| 2 ) + o(^), J G J*. (33) 

Again, from Taylor expansion and Lemma f4.1( i) as well as ifk = o(ip^), 
we get 

f 3 (x k + d k ) = f^) + gj (x k ) T (d k + d k ) + |(4) T VL/i(^)4 + o(\\d k \f) + o(^) 

= /,(* fc ) + ^(x*) r (dg + 4) + |K) T vL/i(x*K + o(||4n 2 ) + (^) 

(34) 

Combining ( 1331) and (1341) . it follows that 



+ (||dg|| a ) + oM. 

(35) 

Thus, substituting (135]) into (|32|) . we have 



go(x k ) T (d k + d k ) = -(4) T B k d k + £ Af/^x*) + |W8) r (V^(s*, A*) 

-vL/o(^))4 + o(H4li 2 ) + oM- 

(36) 

In addition, in view of gj(x k ) T d k = —fj(x k ), j G J k , from (1311) . it follows 
that 



flb(s*) T d* = -(dS) r S*(fi + £ (37) 
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Now, substituting (1551) and (1571) into (15Uj) . we have 
A, = -(dgffWg + £ ^jfj( xk ) + \{dt) T V 2 xx L{x\ X k )d k - ag (x k fd k 

- p (l-a)ipl + o{\\dX) + 0{ipl) 
= (a - lM) T B k d k + (1 - a) £ A^(* fc ) + i(^) T (V^(^, A,) 

-2?*K - p(l - a)<pi + o(| |4| | 2 ) + O(^). 

This together with Assumptions 13. 1[ 14. 2\ X^fj(x k ) < and a G (0, |) as 
well as 9 < a shows that 

A* < (a - \)a\ \d k Q \ | 2 + o(| |4| | 2 ) - p(l - a)<A + o(rf) < 

holds for large enough. Hence, the first inequality of ( I14p holds for t — 1 
and k large enough. The whole proof is completed. □ 

Remark 4.3. In order to overcome the Maratos effect, under some suitable 
assumptions, some norm relations of directions and a weaker set relation 
corresponding to the strict complementarity (i.e., JJ" = I(x*)) are given in 
Lemma 4-1 firstly. Then, by these results, we obtain the Theorem i-e., 



the inequality |7^] ) is always satisfied for t = 1 and k large enough (i.e., very 
close to the solution of the problem). So, the Maratos effect can be overcame 
in our paper. 

According to Theorem 14.21 and its proof for case of j G" I(x*) as well as 
relationship (128]) . the following lemma holds immediately. 



Theorem 4.3. Under the assumptions stated in Theorem \4.%\ we have ipk+i 
after a finite number of iterations, i.e., x k+1 G X for k large enough. 
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At the end of this section, based on Theorems 14.21 and 14.31 as well as 



Lemma H~Tl using Theorem 3.1.3 in 14(, we have the superlinear convergence 



of Algorithm 12.11 immediately as follows. 



Theorem 4.4. Suppose that Assumptions \2. 2\ \ 3.1\ \4-l\ and \4-2\ are all satis- 



fied. Then, \ \x k+1 — x*\ \ = o(\\x k — x*\\) , i.e., Algorithm \2.1\ is superlinearly 
convergent. 

5. Numerical experiments 

In this section, in order to illustrate the computational efficiency of Al- 
gorithm 12.11 some preliminary numerical results are reported, and the com- 
puting results show that Algorithm 12.11 is effective. The algorithm was im- 
plemented by using Matlab 7.5 on Windows XP platform, and on a PC with 
1.99 GHZ CPU. The approximation Hession matrix is updated by the 
BFGS formula described in \}\ . 

During the numerical experiments, the parameters are selected as follows: 

7 = r] = 0.5, 9 = g = 0.4, a = 0.6, £ = 1, ( = 0.2, 
a = 0.3, p = 1.5, 5 = 3, r = 2.5, e = 0.5 3 . 
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19] . In addition, we fur- 



We test some problems which are taken from 

ther test Svanberg problems in different dimensions and with different initial 
points, which are taken from 



Execution is terminated if the norm of dg 



3- 

is less than a given constant e > and (fk = 0. The columns of the following 
tables have the following meanings: 



Prob: the number of the test problem in 



11 



n/m: the number of variables/inequality constraints of the problem; 
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Code: the name of the algorithm; 

NFO: the number of objective function evaluations; 

NIO/NII: the number of iterations out of/within the feasible set; 

NI: the total number of iterations, i.e., NI=NIO+NII; 

NF: the number of all constraint functions evaluations; 

FV: the objective function value at the final iteration point; 

CPU: the CPU time (second). 

Finally, an "— " in the following tables indicates that the corresponding 
information is not given in the corresponding references. 

Experiment 1 (for small-scale problems). For this part, in order to 
show the computational efficiency of Algorithm 12.11 (denoted by ALG l2TTj) . 
we test some small-scale problems and compare ALG 12.11 with some other 
algorithms, and the numerical results are given in Tables 1-4. 



In Tables 1 and 2, ALG O is compared with ALGO [8| and SNQP [15| 



for the same test problems, the stopping criterion threshold e and initial 
iteration points are the same as that reported in [8[ and 15j, respectively. 



From the viewpoint of the numbers of NIO, it follows that ALG 12.11 can 
always enter into X after relatively small iterations. Furthermore, from the 
viewpoint of the numbers of NIO, Nil and FV, the results show that ALG 12. 11 
is obviously better than ALGO for most of test problems. The performance 
of ALG l2~T1 m terms of Nil is better than SNQP except problems 33 and 76. 

Tables 3 and 4 gives the compared numerical results for ALG 12.11 and 
ALG 3.1 as well as ALG 3.2 2l| . The test problems and stopping criterion 
threshold are the same as in |2l|]. The numerical results in Tables 3 and 4 
show that ALG [2J] can always enter into X after small iterations, and ALG 
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12.11 is more better than ALG 3.1 and ALG 3.2 for the test problems. 



Table 1. Numerical results for Experiment 1-1 



Prob 


n J Tn, 


Initial point 


Code 


NIO 


Nil 


NF0 


NF 


FV 




CPU 


012 


2/1 


(6 6) T 

(V, V) 


ALORTTI 

jTL J— J V 7 ^^^^^ 


17 


3 


21 


41 


-3.0000000E - 


hOl 


0.06 








ALGO 


25 


28 


29 


57 


-3.0000000E - 


h01 










SNQP 


7 


12 


12 


29 


-2.9999999E - 


hOl 




029 


3/1 


C_4 _4 —d) T 
V ^) 


ALGl2~Tl 


3 


9 


13 


16 


-2.2627417S - 


1-01 


0.05 








ALGO 


1 


11 


14 


27 


-2.2627417E - 


hOl 










SNQP 


1 


12 


17 


42 


-2.2627416E - 


hOl 




031 


3/7 
°/ ' 


(2 4 7) T 


ALGl2~T1 


1 


16 


18 


309 


6.0000000E + 


00 


0.06 








ALGO 


1 


20 


23 


43 


6.0000000E + 


00 










SNQP 


1 


19 


20 


52 


6.0000089E + 


00 




033 


3/6 


("2 4 6~I T 


AT,0[2~TI 

jTL J— J V T ^^^^^ 


I 


9 


11 


134 


-4.5857864E - 


hOO 


0.05 








ALGO 


2 


16 


17 


67 


-4.5857863E - 


hOO 








fl 4 6) T 


ALG[2~T1 


1 


11 


16 


570 


-4.5857864E - 


hOO 


0.33 








SNQP 


2 


21 


23 


116 


-4.5857290S + 00 




034 


3/8 


(2, 2, 2) T 


ALG|2~T1 


5 


10 


16 


166 


-8.3403245E - 


- 01 


0.06 








ALGO 


8 


26 


68 


198 


-8.3403244E - 


- 01 




035 


3/4 


("1 2 3~I T 


ALG|2~T1 


1 


(-; 


g 


67 


1.1111111E - 


01 


0.03 








ALGO 


s 


11 


12 





-3.4500000E - 


h00 










SNQP 


4 


9 


9 





1.1111111B - 


01 




043 


4/3 


(—10, 2, -8, 5) T 


ALG[2~T1 


9 


5 


15 


95 


-4.4000000S + 01 


0.06 








ALGO 


23 


26 


27 


163 


-4.4000000E - 


F 01 








(0, 2,2,4) T 


ALGO 


7 


9 


17 


135 


-4.4000000S + 01 


0.08 








SNQP 


1 


11 


11 


69 


-4.3999999E - 


1-01 




044 


4/10 


(-20, -20, -20, -20) T 


ALGO 


4 


10 


15 


296 


-1.5000000E + 01 


0.08 








ALGO 


7 


15 


18 





-1.5000000E - 


hOl 




066 


3/8 


(0,0, 100) T 


ALGO 


10 


54 


65 


1067 


5.1816327S - 


01 


0.48 








ALGO 


34 


39 


40 


161 


5.1816327E - 


01 




076 


4/7 


(1,2, 3,4) T 


ALGO 


5 


16 


22 


345 


-4.6818182S + 00 


0.11 








ALGO 


6 


11 


15 





-4.6818182E - 


h00 










SNQP 


2 


14 


14 





-4.6818171E - 


h00 




100 


7/4 


(0,3, -3, 
3,0, 1,0) T 


ALGO 


18 


39 


58 


861 


6.8256637E + 


02 


0.61 








ALGO 


4 


33 


47 


363 


6.8063006E + 


02 
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Table 2. Numerical results for Experiment 1- 1- continued 



Prob 


n/m 


Initial point 


Code 


NIO 


Nil 


NF0 


NF 


FV 




CPU 


113 


10/8 


(4, 10, 10,2,0, 
11,4,0, 12, 10) T 


ALGl2~Tl 


12 


4 


17 


378 


2.4306209E 4 


- 01 


0.14 








ALGO 


6 


17 


21 


205 


2.4306209B 4 


01 








(0, 2,9,5,0, 
1,9,8,-10, 10) T 


ALG[2~T1 


9 


7 


17 


355 


2.4306585S 4 


-01 


0.17 








SNQP 


9 


29 


29 


258 


2.4306211E 4- 01 




264 


4/3 


(8, -5,6,-4) T 


ALG[2~T1 


18 


5 


24 


142 


-4.3987578E - 


f 01 


0.13 








ALGO 


19 


26 


27 


161 


-4.3999999E - 


f 01 








(0,0, 0, 10) T 


ALGEHl 


17 


5 


23 


146 


-4.3987578E - 


f 01 


0.16 








SNQP 


4 


16 


17 


122 


-4.4113405E - 


f 01 





Table 3. Numerical results for Experiment l-II 



Prob 


n/m 


Initial point 


Code 


NIO+NII 


NF0+NF 


NDF0+NG 


CPU 


Rosen / Suzuki- 1 


4/3 


(0,0,0,0) T 


ALG A 


04-17 


18+117 


18+54 


0.09 








ALG 3.1 


76 


1185 


308 


2.88 








ALG 3.2 


77 


1225 


312 


3.04 


Roscn/Suzuki-2 


4/3 


(2,4,8, 1) T 


ALG A 


9+10 


20+120 


20+60 


0.09 








ALG 3.1 


68 


1034 


276 


2.36 








ALG 3.2 


55 


793 


224 


2.06 


Wong Problem- 1 


7/4 


(1,2,0,4,0, 1, 1) T 


ALG A 


04-24 


25+228 


25+100 


0.16 








ALG 3.1 


157 


17756 


790 


21.48 








ALG 3.2 


157 


17759 


790 


21.54 


Wong Problem-2 


7/4 


(3,3,0, 5,1,3, 0) T 


ALG A 


94-48 


58+624 


58+232 


0.38 








ALG 3.1 


171 


18677 


860 


22.48 








ALG 3.2 


151 


16921 


760 


20.88 


Quadratic Problem- 1 


2/2 


(-0.3,0.0) T 


ALG A 


0+7 


8+38 


8+16 


0.03 








ALG 3.1 


48 


292 


147 


0.76 








ALG 3.2 


49 


301 


150 


0.76 


Quadratic Problem-2 


2/2 


(2.2, 1.6) T 


ALG A 


6+4 


11+42 


11+22 


0.06 








ALG 3.1 


50 


314 


153 


0.72 








ALG 3.2 


43 


286 


132 


0.68 
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Table 4. Numerical results for Experiment 1 -II- continued 



Prob 


n/m 


Initial point 


Code 


NIO+NII 


NF0+NF 


NDF0+NG 


CPU 


TFI1 Problem- 1 


3/1 


(-10,0,0) T 


ALG A 


0+12 


13+32 


13+13 


0.05 








ALG 3.1 


35 


2663 


792 


1.46 








ALG 3.2 


35 


2663 


792 


1.46 


TFI1 Problem-2 


3/1 


(1,1, 1) T 


ALG A 


1+7 


9+17 


9+9 


0.05 








ALG 3.1 


23 


2301 


528 


0.98 








ALG 3.2 


22 


2289 


506 


0.96 


TFI2 Problem- 1 


3/1 


(2,2,2) T 


ALG A 


0+6 


7+23 


7+9 


0.04 








ALG 3.1 


30 


1343 


682 


1.04 








ALG 3.2 


30 


1343 


682 


1.04 


TFI2 Problem-2 


3/1 


(0,0, 0) T 


ALG A 


4+7 


12+25 


12+18 


0.06 








ALG 3.1 


48 


2129 


1078 


1.58 








ALG 3.2 


48 


2135 


1078 


1.60 



Experiment 2 (for middle-large-scale problems). Considering the all of 
tested problems above are all relatively small, we further test the Svanberg 



problems 



20] problems, some of them are larger and therefore interesting. 



The experiment results are given in Tables 5-6. 

In Table 5, the performance of ALG l2TT1 is compared with SNQP, ALGO, 
FSLE 22[| . The initial iteration points and the stopping criterion threshold 
are the same as that reported in [22j . From the results in Table 5, in viewpoint 
of Nil and NFO, it follows that algorithm ALG 12.11 performs better than 
FSLE, SNQP and ALGO in most cases for problems Svanberg. 

In Table 6, we further test Svanberg problems (in different dimensions) for 
some infeasible initial points, and the stopping criterion threshold is e = 10 -6 
and ifk = 0. The results show that our algorithm ALG I2,ll is always successful 
for all cases, and the iteration points can enter into the feasible set so faster. 
In view of NIO, Nil and CPU, it follows that our algorithm is effective. 
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Table 5. Numerical results for Experiment 2-1 



Prob 


n/m 


Initial point 


Code 


NI0 


Nil 


NF0 


NF 


FV 


CPU 


Svanberg-10 


10/30 


(0, 0, . 


...or 


ALG|2~T1 





16 


17 


1140 


15.731517 


0.34 










SNQP 





28 


28 


1753 


15.731533 


_ 










ALGO 





15 


21 


1050 


15.731517 













FSLE 





36 


227 


258 


15.731517 


_ 


Svanberg-30 


30/90 


(0,0, . 


■■,0) T 


ALGO 





25 


26 


5490 


49.142526 


1.77 










SNQP 





27 


27 


4975 


49.142545 












ALGO 





26 


38 


5670 


49.142526 












FSLE 





101 


777 


864 


49.142526 


_ 


Svanberg-50 


50/150 


(0,0,. 


■■,0) T 


ALGO 





33 


34 


11550 


82.581912 


5.91 










SNQP 


o 


37 


37 


11762 


82.581928 












ALGO 





35 


51 


12750 


82.581912 












FSLE 





108 


881 


968 


82.581912 




Svanborg-80 


80/240 


(0,0,. 




ALGO 





42 


43 


24720 


132.749819 


15.38 










SNQP 





47 


47 


24100 


132.749830 












ALGO 





47 


68 


27360 


132.749819 












FSLE 





190 


1666 


1835 


132.749819 




Svanberg-100 


100/300 


(0, 0, . 


..,o) T 


ALGO 





46 


91 


53700 


166.197172 


26.38 










SNQP 





46 


46 


27880 


166.197199 












ALGO 





53 


66 


35400 


166.197171 












FSLE 





178 


1628 


1782 


166.197171 
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Table 6. Numerical results for Experiment 2- II 



Prob 


n/m 


Initial point 




NIO 


Nil 


NFO 


NF 


FV 


CPU 


Svanberg-10 


10/30 


(10, 10, 


, . . . , 10) T 




3 


15 


19 


1320 


15.731517 


0.28 






(-10, - 


10, , . . . , - 


10) T 


2 


16 


19 


1440 


15.731517 


0.28 


Svanberg-20 


20/60 


(10, 10, 


. . . , 10) T 




4 


22 


27 


4380 


32.427932 


1.11 






(-10, - 


10, , . . . , - 


10) T 


3 


24 


28 


4560 


32.427932 


1.23 


Svanberg-30 


30/90 


(10, 10, 


,...,io) T 




3 


25 


29 


6480 


49.142526 


2.33 






(-10, - 


10, , . . . , - 


10) T 


3 


24 


28 


6030 


49.142526 


2.50 


Svanberg-40 


40/120 


(10, 10, 


, . . . , 10) T 




3 


28 


32 


9480 


65.861140 


3.67 






(-10, - 


10, , . . . , - 


10) T 


3 


28 


32 


9480 


65.861140 


3.58 


Svanberg-50 


50/150 


(10, 10, 


. . . , 10) T 




14 


26 


41 


24150 


82.581915 


7.84 






(-10, - 


10, , . . . , - 


10) T 


1 


34 


36 


12900 


82.581912 


5.75 


Svanberg-80 


80/240 


(10, 10, 


. . . , 10) T 




2 


43 


46 


42720 


132.749820 


17.09 






(5,5,.. 


,5) T 




2 


47 


50 


63840 


132.749824 


19.53 


Svanberg-100 


100/300 


(10, 10, 


. . . , 10) T 




3 


43 


47 


57300 


166.197173 


26.81 






(5,5,.. 


,5) T 




2 


62 


65 


112500 


166.197178 


40.55 


Svanberg-150 


150/450 


(10, 10, 


. . . , 10) T 




40 


44 


85 


227700 


249.818369 


130.41 






(5,5,.. 


,5) T 




3 


62 


66 


123750 


249.818369 


96.16 


Svanberg-200 


200/600 


(10, 10, 


. . . , 10) T 




4 


78 


83 


219600 


333.441310 


279.95 






(5,5,.. 


,5) T 




2 


84 


87 


236400 


333.441310 


287.45 


Svanberg-250 


250/750 


(2,2,.. 


,2) T 




1 


85 


87 


275250 


417.064989 


602.78 






(3,3, . . 


,3) T 




1 


90 


92 


281250 


417.064989 


593.48 



Experiment 3. To show that ALG [27T1 performs cycle I for most of the 
iterations, we give Table 7 below relative to Tables 1 and 2 for test problems. 
The row labeled fj lists the problem number as given in Tables 1 and 2, and 
the column labeled (j lists the number of cycle I and cycle II performed by 
ALG |2TT| , i.e., N-cycle I and N-cycle II, respectively. The results reported 
in Table 7 are encouraging. Obviously, N-cycle I is much more than N-cycle 
II. Especially, for problem 12, ALG 12.11 always performs cycle I and does 
not perform cycle II. This also illustrate Remark 12.61 from the viewpoint of 
numerical results. Thus, the cost of computation for ALG 12.11 is relatively 
small. 
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Table 7. Numerical results for performing Cycle I and Cycle II 



tt 


12 


29 


31 


33 


34 


35 


43 


44 


66 


76 


100 


113 


264 


N-cycle I 


20 


8 


15 


9 


9 


6 


12 


7 


52 


17 


49 


11 


22 


N-cycle II 





4 


2 


1 


6 


1 


2 


7 


12 


4 


8 


5 


1 



6. Concluding remarks 

In this paper, we propose a new algorithm of combining (QP) subproblem 
with SLE for solving nonlinear inequality constrained optimization problems. 
The new algorithm starts from an arbitrarily initial iteration point. In order 
to ensure the global convergence of new algorithm, the search direction is 
obtained by a convex combination of the master direction and an auxiliary 
direction, which are solved by subproblem (QPs) and SLE (jH]), respectively. 
For overcoming the Maratos effect [5|, a higher-order direction is obtained by 
solving another SLE (TTTT) . Moreover, the iteration points can always enter 
into the feasible set X and only one SLE need to be solved after a finite num- 
ber of iterations. Using line search instead of arc search, our new algorithm 
possesses global and superlinear convergence under some mild assumptions 
without strict complementarity. Finally, some numerical results show that 
new algorithm is promising. 

As a further work of this paper, the techniques introduced in this pa- 
per can be extended to solve general constrained optimization problems and 
minimax problems. 
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